(19)H*H4MW UP) 02) & H # It & $g (A) <ii)W!fr(i«&W#* 

#11^10-42282 

(43) l^BH 0 ¥ffclO*P (1998) 2^130 



(51)IntCL s W&m^ ^W«E3ffi»^ F I 

H04N 7/18 H04N 7/18 U 



« Sit3»»a*8 OL (it 13 K) 



(21) yttSHilr^j 



(22) /.L'iIkh 0 



M¥9- 102160 



sp^ 9 4f (1997) 4^180 



(31) «5fc»£3K#^ (Mpp8- 101230 

(32) mt 0 ^8 (1996) 4 J§ 23 0 

0* (J P) 



(7DHSBBA 596056081 

iXra 

WJ561 4 WJB»rX*-f > 

210*1 

(71) WIS A 596056092 

m 

AEi&"*BaittUJBB4tl5#l -515-Jf 

(72) ^69i# hjp ft£ 

WT561 #36 £> 4 MBWTX ^T»3> 
210^ 

(74) A 2f*± MR 3£f£ 



(54) K«C?>«*] *«*g^>XirA 



(57) 

] 



[jg^Sl ^ l J =3 y ^ - 30tC»t8 Lfc*;* 531TH 



tJBr!?! 





/- — * — k 



i 1 



1 



36 



3 2[-feV-tH 



3 1 (£-><5) 




'1 



i - 



' 



3 3(itfiJbS) 



(2) 



^m^ 1 1 0-42 2 8 2 



f&at?Hft*«6£b-rststh#sts 3;*^oittEix--£ 

WS^en, K*f^^0 3^7cffiH, 3 3KSS3e&t>TS 
^^£t-£®*^l§<^ MiBfi^*icli;i6*u 3* 

KliiiStu !5£;><^<7>3^ffiB, 3 *7LSl»aif H 

ft*tttbrstttB#iB!t, luiB^T^^ii^env mis 
£ £ f a Wfitff ?s ^> x t 1 /ao 

«^ 3*^ittH^-**tt)WLT^Stt*ft^Si:, K 

L fc JftE x - * * mi IB * / 5 T^»»«Mft ^ft^a^T 
[ftSRlI 5 ] f? ISf 53*ftft*t/uT^ ^ 5 THIS L 

# L fc Jti El x - # iu 8 3 * * 3 V<D m I B fcTfe % «■ ^ fe li 



1 3t6 CD WHS US* isXT-L\ 0 

im&m s ] tjiB^ai^soaaijieso^--* t Btea 

* ^ ^ ■?? (DiiU «M» x — £ t =£ IBfSf & #ia * M lefl 

mmcomm^mm] 

[000 1] 
[0 0 0 2] 

fcWSxUtr^a^ (TV) ffiilT?fcJ\ BggtftfKS-ft 

t 7 ^ ^ > -y— ^3Egj»o)tftia«reA. s tp«#ffl^±steTfe 
^fco fits ^<o*flK#ft\ vy^z^ix-cmm-z 

[0003] Lj^L&tfSx citieco— iic^astiffiiitc 

* r ^ft*Sco««*ffiTTSTiE-r^i:l^«lffi3b^ 

[0 0 0 4] 

LfcMH<h Lin mmz&v^&mm&^LTL 
[0005] dcDcfc^^raatt. TvsjicQSss^ 



(3) 



RfFJ^F 1 0-4 2 2 8 2 



I — t 



[0 0 0 6] iC5Ts GPS (Global Positioning S 
J** h ^ LTV < C<t^RjfgT^^.6\ U7 7 ;!/ 



rooo7] ^mmitmt^mm^M^zt^ntc^cD 



[0008] ^mmoiftommt, mmmmcnr^iE 

[ooo9] *^0iicTOiwit 
sis, mmic^f^itimmi^^m^n^m^m^P 

[0 0 10] 

3 * tc^H] 7= — * * iflf mm Hi *»to«6ate*KB*S* 
[0 0 1 1] Sa?3l2tC^*B*«a^>X7 1 /xttv fRW? 

#® t t*S*ft * *i T 3 12 x - £ * MfB^ Hi ^ 

[0012] m^m3 im^mmm^yx9-u^ 
Sit, ffiimftmcffizzn, mtEti^^omm^ 



t&m^&t. is»tt#faic»w*nT^S3*5tcoiftiii 

[0 0 1 3] fi^<Jl4tC^^^fa^^XxA^. ^ffi 

3^7^0**11^^ 

[00 1 4] ff^StC^^W^S^^XxA^, *£©J 

r*»ft**«^T**>«57?a»Lfc^*a^?--S'> 
3;^7cffiM, 3*7Die»ayBift*tttti , r*«tu^s 

is 3*SOittEax-***lttLT^*«»#l8:iv IS 
tSJft^S K * ftT l ^ 3 ;*7c OiftH x - £ =£ SulB« 

[0015] m^meimzm^^i/Zs^ixte, 

«?t ^ a ^ ^ & c t s-tsia i -r 
[ooi6] wa<Ji7tc^^^^s/xx/^^v isa? 

^aiestT^^^^TSsct^afrso 

[00 1 7] lf*J18lCflRSB^a^v'X7 1 ^t*. 
J11 -7^pJtl*HCfct^Ts fuIB^m#KCD^l±S^O 
x - * i MSB * P« 5 7?<7)a«RSMfc<7) x - * <t S: IB»"T ^> 

[0 0 18] ^B^^^JI^vXxXaTI*, ti^'yO 
WCDiti^at^W, BEPi:35:Saig*!fflf(7)±ftEl1f¥B*s 

[0019] ^-r^^y^tiv grfii, 



(4) 



1 0-42282 



[oo2o] ^^^icm^zmmm&m&^mnwizm 

[0 0 2 1 ] &&woi&o^^*s Ixl*. ±hsl 

7Lffi«*ra3£r*ffiBiRiE*at^ mzntdiiwmm 
■««#5S«sn*£\ ^(Diirmmmzjtczmnwz 



[0 0 2 2] ZftZ<Dmm%^SXTl±iC33^ 
[0 0 2 3] 

[0024] em it^mmom^mTT^yx^iAcom^ 

7jk?1M&mZ%>V, ^UpX£— SOlZtillKJlO**^ 
31iu ^FS^^^a^Mf ^> ^ p-^31 CQ 3 ^teIiES, 3 

"\ 'J P X£ »303t?fRtr UfttfSs * ^31 tCT^S^l^ 

W^5r-£iHz>+*32KT«M^ 

£*? v iS^±ft±tc:iRj^TJ^«*n, itt±J^33(Z)§««34 

TSff^tl^o Tj/^W^C G (Computer Graphics) 

ZL--j h35tC^en^>o CGl-'V h35[CT, ^»IES 

ll] IS ^ ~ ^ ^ 5^ V X 7 J U ^ 3 6 1 C ^ 6 n ^ , , ^Tcs 
LTs rVxyUY36^ ^31 Tl# 6 



[0 0 2 5] S/c, i«±^^T"T^Sa^37tr^ ^Jx. 

s ej ^37 co 3 ^jcfegtf$g co z m&±m 33 t zm s n 
^■o)a^itirsg}^37^ffi"rn^; fvx7u^36 

t ^ S B m 3 7 CO S 7b^ ifs ^ tl ^> o *£<D&Wim7 

a^nTt^ll¥Wt^ [ t 3 Tgi^]^37^^ll^T : ^^ Sid 
[0026] ^mmo^mm^yx^juomi^m 
(m 1 <nmm<7>Bm) But*. *«j»<di^«3S5'X7 ; 

a /\'JpX#--, ^Hco^T«:tc^tt*ti^g|3« 

[0 0 2 7] fH?T<ttCtt. Tj;*^ 1 A^7103* 

»5 r -*tca!»-rs«*aE*»4i:, Hftfisatiflixvp 

5«fc, ^^^1COfwM«l±iffl(7)GPS^m6<h, GPS 

-j>t: 0 ;i~£23<!:. rtap*fctt^aSO)«B1*ai*24i, IB 
[0 0 2 8] A* ^114, — iKWttTV/j^^Sfctttt 



[0 0 2 9] *y^HCt*. ^C0 3 35d7ciS»*«6Eti-rS 
fc»<7)^>^PH2>^23^IXtl^6nTfcy, 
p-feV9-2ti. *f^^ 1 co^\'y Kft (TJ^^ltD^Hffi 

co«m#^ : &if^^4^\m^^^ 0 «*ssss4 

Kffiff^^H P (H (Head) 

#[£jx~<£, p (Pitch) : 7b ^ 1 (D±TJ5\v)Qi^? 
— B (Bank) : 1 cD^^f^^fti^T 1 ™^) 

^ L T ©J =] > e ^ - ^ 9 -\ * ^? T ^» o CC0J;5 

tU£v ^"-^x->^4±i^v f >'<P5xdt, hUV^/U- 
i»/\°>tt0TANS Vector ™W&&mZ1*%>* 
[0 0 3 0] Sfc. 1 C0UVXtC«S , T*X-jU 

U V ^IC tts Hift^ttiffi x > P - 9 3 ^ y ttlf 6 tit 
J?Us ^3^^ 1 (DX-ZxUVXcOIsHteft^aiLTfiSttl 



(5) 



1 0-4 2 2 8 2 



i§^FVf-^ (FV (Focusing Value) :.t)./7l 

[0031] MtCv *><5 1 coifi^a, ^-try- 

S£IB6te N Z>^7;6 N 6<D §fit§^§- (GPSt- 
$0 lCiSeoT*P<5 1 (7)3*7LS6Wffia*5Rtf)Ts XY 
Z • D R ♦ DTf-^ (XYZ :^^71 (DitLm^— 

It-^^^I LiaU, GPS8B6H CCDffilEx 

tLTttv WI*J£\ y--*±WlPS-3000, MJ>y7l4± 
OSierraTM GPS^y^-try hf^ffflW^o G 
P S 8 tt, sefflWftS«+t-tfXS-SI*« 

IMftf taoTAS < tf^^G P S r-^ 

[0 0 3 2] U±(D^otsi>^^a^y^-2 t Hftfijaj 

IlV3-?3, 4, fflTOSSSS, GPS 

[0 0 3 3] ©JtP^=]>b 0 a.-^9«, ±izEL/c^^5 
ffiH-SBftH-WSBT^IiStlfc/^/-^ (XYZt- 
HPBf-^, FVf-^) fr6> 3>t 6 i-^ft 

ffl-TSc Kaj-ra*fljSffiB©/\ o: 5^--^tt. #;><^icd 
(ttB/<5^-*<tra*K, ftj^diffi (x, y, z) , *i 

fft^rp) CvyKftH, t°7fftP, /t>^flB) v lift 

(FV) CD 7 *7LT*3£«S'tl* 0 SiJiP^^^b 0 ^-^ 9 

rct&mmm ows, miBj^stfljiff) &^x£tvio 

[0 0 3 4] ^E-^TVIOti. »6«Jfl*SB€-|*|j8LT 
Us 13*=? 1 Tl#6tlfc^f^$iJ»fflD>tfa-~^9 



/ci6v GU I (Graphical User interface : ^-y~P ^ ii 

[0 0 3 5] £fc\ ftJfiTO 9 t£ v *I,£ffiB 

CO/^^-^x™^ (XYZ, HPB, FV) £±*!}<D 

vzi-^n^tij-r^o ^--?x>zi»^ntt, suapffll 

[0 0 3 6] *JWfflZl>t;jL-^9tt, flffrWC 

2, Bflfil!±Jfflx>=]-^3^CD«JSHiaS, K3H»iES' 

[0 0 3 7] g«SB2n*. 2&fSliB123b N 65HffiS*n/c 
■SStiWfcSffiU ^-#x^-#22&tfBB»*B25'\ 

x-^t?l^ffiMc0/^P<~^^»~^ (XYZ, HP 
B, FV) StfDR • DT^-^tlCffl-^-U Wix- 
£#W^ffi»»B26^iiiU ^cD/^y™^^-^ 
(XYZ, HPB, FV) MDR*DTr-^^*X 

h=)>tfa~^23^ffi-r^o cKD^-^xyzi-^ll 

VX-50T/R£1£fl3"r*So 
[0 0 3 8] tah3>La™^23^ A^tlfeM 0 ^ 

(XYZ, HPB, FV) frS* =J VtfzL 
- £ F*3 (0 ffiS CO 3 &75ittEI ^ t C J5 * *I CD / \° ^ ^ ™ 
^^»iil"r^o Si±11-^?l^McD/\ 0 ^^™^tiv ±a 

Zl Vtfo.-^23^ CD-ROM, HD, MOr^^ 
»6^6a«^a5IB1t«Ett:24lc|Btt LTl^S 3*56^*60 

Ifi«l f tt!M r mSS, JrIJIL JSSS, OjM) ^CGW 
«x-^tLTWl««l*SB26's3SJtti"r*o £<D3$Z7t 

<Dmm 9 ^ ^ins^JtiT ^> fc^cD y ^ b o x t 7 <t t t 
« % *"jfflifflzivtra."^9i:ra««c. y^x^^tttiy 

7h^x7 FVPXj ^ffifflT^^o 
[0 0 3 9] K«llB*SB26«. ^-^^=3-^22^6 
(Dm^Ltcmm?-* t, *Xh3>ta~^23^6cO 

«^-^SWHSillttS^BSfc«VTR^tli^-rSo IBS 
SB25fr*, SfiSB21TS<s*tlfcft^«: e^^-^ 

J ijr r — ~~j £3. -d~ 

[0 0 4 0] :*tc, S»mc-^^TKP^-r* D S-Tx i» 

-rn-te>^2 f Hft«uufflx>=i--^3^<7)«3Wsas 

f ?x7 7 tCcf;y^T^n^o ^y<^ i T^*^eift±7trig 



(6) 



ftmW- 1 0-422 82 



n — £ 9^\tti^J^tl^ MUU ^7/^ 1 CD3^^ffiS^^ 
[0 04 1 ] *iJffll«=iyfcfa-*9tc:*30^Ts Ctl5<7> 

stus-nSc ^ Lis a^^i ra^fftifcWiistciircD 
mmm^Mm Lzftzm&ft*-* t v lotca^n 

[0 0 4 2] viotca^*nsH«io 

TSSVS'JiTl^ f IT, /Way b&TcteWtmmoy 

[0 0 4 3] Sfc* WiSffiBcO/^^-^x"* (XY 
Z, HP B r FV) tDR • DTT"*ii)^ #J»ffl3 

[0 0 4 4] Jti±CD»5&ST*M\ ^tt^6CDm^fi-§- 

(XYZ, HPB, FV) MDR • DTr 
[0 0 4 5] *X h=l>tf^-^23W^ i^* — &T 

(xyz, hpb, fv) ^A^irn, cntcio 

* (7) 4®jS 6> 241*81 * *ix 6 tl fc tttEl If ¥ B WtMS 



[0 0 4 6] mmmMmw26^t7i<nm&?-$iD <: r- 
^x-^^w^ab^^/c^vTR^\ij^*n^o v 

[0 0 4 7] Sfc* c(7)^WlT-^tC^S1-^«i 

, Tuwj**o*fftciiPATs mmwu (cm) * «sb<d* 
^ (Dai, eoj, fjil G^) cowstsaLTa^* 

[0 04 8] Sfc, SffiS»21T««*ftfc\ WHftx- 

£>ffi^-#\ fBSSS25tc:t3JI6tiTi3»*n^ 0 IBS* 

[0 0 4 9] ^(Di^SW^JS^^^^Tlix 



[0 0 5 0] $/c, C©«Ml!S/^>XTix(*s ittMW^: 

[0 0 5 1 ] IXTs *560^lci5UNTMSlS^Sl/^'JttPffl 
[0 0 5 2] $-T\ 0 5=&#gBLT«J^»lffH:^T!ft 

*?T-otc& (SI), 



(7) 



^Ifflspi 0-4 2 2 8 2 



(fPTfrT*-*, aag^x— S-ffl^iit? (S2) 0 
#^T\ Jj*^ 1 <DT-$%mmitLtc& (S3) , 1 
*j£©*f**A*|-rs (S4) o CC?, S*jS£LT 

tts a«*8jasfct*sB»tcsttai$s«)ti!3*if. "torn 

^is^-rs (ss) 0 a^iox—s* 
[0053] ikiZs me^mMLTmmommmmz^ 

TZt^tl&mttlLZ (S11) , ffi^LTfc^fcl^BU 

^A^^-^A^e^iuaiffli^itB-rs (si2) 0 ^mm 

[0 0 5 4] — * % K*IIEHrt^»S»$tttt, 

^><^ 1 <7)A^7^"-*€-*ia0 1 ^tu^A^x-^fcL 
T«#"TS (S15) <, CGB*«^-£ffiiIi£M' 

«iLTV-h f JXh^Mn (S17) 0 Kffi«^<fc 
ifi5fira«(DW$g*»«ft Lfc Z / W 7 7* «t <Drt§=& 
Lfd£ (S18) % iffl^^tcS^TfSBteil^^^ 



[0 0 5 5] ±JiELrcfiaJIi&^tCfct^T. 
fllS tl I C S *3V T ittll x £ «■ JtSHS "T * <fc 3 ic L T I ^ 



*- /j^^lojaUBWffclciElUc^aL/cCGil&Hx- 



[00 5 6] (SI 2 CO^CDffmi) ±^CDm 1 cD^ffiO 
^ X £ T V 1 0Tfc JIS C £ fi^mr* LTc t\ C<7) 



E3 



r>X7A (*2<0SSfi<iWB«) <D«f8m^T«5£E 
[0 0 5 7] H7^45lNTBI2i:ra— Sf^tcttra— = 

•fe>+J-2, Hft«aifflx>3-^3 f «*a»»4 r « 
^«»5, GPSgI6, ^v^^so-gpsi^ 

SlEgK8U:T«J«*ft£;t>;* =?\iLW. • s6»fitailKSfr 
[0 0 5 8] gf/c, E7fC33^T*X h 3 > t°:i-£23 

2 2 Tffl-t £ tl fc * ¥ 9 ~ $ 1 1)< * X h =J > tf a. - 

[0 0 5 9] S*>, ±iaiLft»1, ^2CDH35SCD^T 

«CDH*® a ffl x > 3 - ^{»T* 3& S o 

[0060] (m3<D$m<DBm) ms\t. ^mmom 
«i^>xta its 3 <Dm&<DBM) om^m^Tm 

jCHTSSo W3H]5ft«!ll*, AFeI, n«r, fflffl, 

[0 0 6 1] H81C3d^Ts 18Stf19ti, GPSgIS 
tF^l^TfcU, cnScOGP SS«18ti^fg1«19i;« 
-WfcLfcP^r-^20^AF^, ffiUfi, mfT 

Ofwlf-^ (XYZt-*) *3lfi1«19tCJ:ySt1fi® 
[0 0 6 2] #ltc, KmcolNTSiP^TSo ssi, ^2 



(8) 



^mW- 1 0-4 2 2 8 2 



fc □ <r- * 20(Dffl»«t B1S 1t*H £ tfgtfijg^ ~ $ T V 

^-^2O0^f^^m^#^K*^TX™^7 ? ^^-rn 

[0 0 6 3] C(7)«t5^LT»*W^«iRLT»^L/c 

[0 0 6 4] C©cfc-5a*fft*Ii:SE*"r*»3R, S6 

I c *5 * H¥K« x - £ <b Ji«J *fl= <7>; \7^^ 

x-* (XYZ, HPB, FV) T&ZfD R • DTf-$ 

[0 0 6 5] m9li, M3 0miMmm(DmMMi7r^y7, 

V10*»«LTfi*v JRfrftrtO^-^TVIOTtHLh 
0^E-^T V27£:Ir)«HC* P 20015^ £JIttftfB 

[0 0 6 6] ±JELft:Bi0^T*t* 1 ^^ttt"****!*^ 

[0 0 6 7] ft 33, ±^L/c&*fi£^!KT1*ffi*T*0 

SuSE<7)^y^1, v>^n-tz>^2 / Hft^tiiffix>=] 
-~?3MGP smW6%m7M±ffit LTfctftl ±a$ 
L/c**SfcO!>jeJB4:lRia»c, **Tl#Sft*»KB*«fc 

[0 0 6 8] ±aL/c*«*OJB»T?«l*RW« 



[0 0 6 9] * 0^ CO tf ^ S ^ V X ^ A T 

Willis iiJBMk 3l««*Rk ^>K^—?\ OjK* 

zkil, K^BS*4>S*»flt 

S£ Wiaasipo j: ^ ft * ^ av?-^i- <Di%m 



A <7)f UJE ffiH =& K ffiA T* * & o 
[0 0 7 0] J-XTs *«Wfl!>3pJffl»S^c■^l^TK*f 
So *^^cD->X7 1 A^3-~tf(t LT^. fiA, 

TV*BM»*Strttlffl-9---tfX*S4:LTa)TV (Augm 
ented TV) "\<fcSErtrff d t^Rlffi?**, 
[0 0 7 1] ^cOctdft^l (&fiJJS#»<&JM*09£LT 

at ajiJk*no^w) (01***1111, Ett&tfswiciipA 

[0 0 7 2] S/c, *^0^CQ->X^A<7)m2^<Jffl^i? 



«±T4aM;wtJ:*cD»»fto 



if tc % o T *a/£U^Uffl # pTbB^S ^>o 
[0 0 7 3] 

[01] *^0^0»«s^^xx^(7)«^ ; &^"r*ixeii 
[i2] ^p^owftffis^xx^o—ffyowsj^^^' 

[0 3] fWfiWc»K*nfc^i:*TVT?CDia«oa^ 
[04] RJBfflOD^Utr^a VT^lj^cOS^J^^T 



(9) 



nmW- 1 0-4 2 2 8 2 



[07] ^^^OB^S^^X^^Oftt^JOWJ**^ 
[El 9] ^^W^a^X? 1 ^ 

1 t//^ 



3 

6 

9 

10 

12 

18 

19 

20 

21 

24 

26 
27 



G P 

*ijffliffl=3>tfa— 5e 
GPSg 



=EZ-% T V 



[mi i 



[05] 














- 







S 1 



S2 



S3 



S 4 



S5 



3 3[«th^l 



(10) 



^mW- 1 0-42 2 82 



GPS 




^ ^ ^ ^ !_HPB[ jga^jftjJl/ 4 




— — cL 



^-3? TV 




o 



XYZ < HPB 



FV • DR * DT 

XYZ-HPB 
F V * DR * DT 



— r~ 




-I 1 



■ ■■■ inn 1 1. 1. i ii i ■ii^^jHfiinrr tt* 

\ 2 2 



25 



XYZ'HPB-FV-DR-DT 



* 

2 6 £=ij- 



x — 4? 



[0 3] 



(11) 



ffiMW- 1 0^422 82 



IH4] 




[H6] 



S 1 1 



S 1 2 




S 1 4 



S 1 5 



S I 6 



S 1 7 



_L 



i|feKlx--£0>ffili 





!>SM3IJK7 







S 1 8 



SI 9 



S2 0 



(12) 



nmW- 1 0-4 2 2 8 2 



[El 7] 



7 ^-^C^ 



6 





I - DT 






-J. „ 






XYZ 
F V * 


* HPB 
OR * D 




XYZ-HPB 
F V ■ DR - DT 



2 



12 MKJW5») 

22 



2 4 H 



2 *3 



pi 



27 



=E_-5?T V 



[El 8] 



7 



GPS: 



8 



1 


? ' DT; 









HPB 



iiUEUS 

. : 1-. , J 














>. 



XYZ * HPB 
F V • D R • D T 



2 



1 1 
22 



25 — 



XYZ - HPB 
FV-DR'DT 



± 



j. : : - 

l ) 



23 




I 



24 



^27 



GPSH© 

~vT"- 

20 18 



(13) 



fflH^I 0-4 22 8 2 



[19] 




XYZ ■ D R - DT 



TT 



iifflx— # 



XYZ * HPB 
F V • D R • D T 



— " i " — ^1 1 




H2 MM(mm) 



XYZ-HPB . 
F V ■ D R ■ D T 



I 



23 



1 9-r 1 



— i 



24 









TV 









gpssb 



20 18 



7PV h^—^i!?- 



(72)»W3# ;f# 

^Bg^Baiuajfflbiss i — sis^- 



i 



PATENT ABSTRACTS OF JAPAN 



(1 1 publication number : 1 0-042282 

(43)Date of publication of application : 13.02.1998 




(51)Int.CI. 



H04N 7/18 




(21 Application number : 09-102160 (71)Applicant : YAMAGUCHI SHIGEYUKI 

SAWAI TAKESHI 



(30)Priority 

Priority number : 08101230 Priority date : 23.04.1996 Priority country : JP 



(54) VIDEO PRESENTATION SYSTEM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a system for superimposing accurate map 
information for photographed video images to the photographed video images and 
presenting them. 

SOLUTION: The photographed video images are obtained by a camera 31 loaded 
to a helicopter 30. Photographing conditions including the three™ dimensional 
positionthree-dimensional posture and viewing angle of the camera 31 are 
detected by a sensor 32. Video data obtained by the camera 31 and the respective 
kinds of the photographing conditions detected by the sensor 32 are transmitted 
to the ground and received by the receiver 34 of a ground station 33. In a CG 
(computer graphics) unit 35threedimensional map data stored beforehand are 
perspectively transformed based on the photographing conditions and CG map 
image data are obtained. The images of a form in which the map information is 
superimposed on the actually photographed video images obtained by the camera 
31 are displayed at a display 36. Alsofor a traveling automobile 37 to which a CPS 
with a builtin communication function is attachedthe automobile 37 is visually 
recognized and tracked in the actually photographed video images on which the 
map information is superimposed. 
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CLAIMS 



[ClaimCs)] 

[Claim 1]A system which presents an image photoed with a cameracomprising: 
A detection means to detect a three-dimensional positiona three-dimensional 
postureand a field angle of a camera which takes a photograph. 
A storing means which stores three-dimensional map data. 

A means to carry out transparent transformation of the three-dimensional map 
data stored in this storing means based on a detection result of said detection 
meansand to gain map data according to a photographed image. 
A means to display gained map data with a photographed image in said camera. 

[Claim 2]An image presenting system comprising: 
A camera with which an aircraft is equipped. 

A detection means to prepare for this aircraft and to detect a three-dimensional 
positiona three-dimensional postureand a field angle of this camera. 
A storing means with which said aircraft is equipped and which stores three- 
dimensional map data. 

Prepare for said aircraft and transparent transformation of the three-dimensional 
map data stored in this storing means is carried out based on a detection result of 
said detection meansA means to gain map data according to a photographed image 
in said cameraand a means for said aircraft to be equipped with and to display 
gained map data with a photographed image in said camera. 

[Claim 3]An image presenting system comprising: 
A camera with which an aircraft is equipped. 

A detection means to prepare for this aircraft and to detect a three-dimensional 
positiona three-dimensional postureand a field angle of this camera. 
A transmitting means with which said aircraft is equipped and which transmits data 
of a photographed image in said cameraand data of a detection result of said 
detection means on the ground. 

A reception means which receives send data from this transmitting meansand a 
storing means which stores three-dimensional map dataA means to carry out 
transparent transformation of the three-dimensional map data stored in this 
storing means based on a detection result of said detection meansand to gain map 
data according to a photographed image in said cameraand a means to display 
gained map data with a photographed image in said camera. 

[Claim 4]An image presenting system comprising: 

A camera formed in a regular position enabling free rotation. 

A detection means to detect a three-dimensional posture and a field angle of this 
camera. 

A storing means which stores three-dimensional map data. 

A means to carry out transparent transformation of the three-dimensional map 
data stored in this storing means based on a detection result of said detection 
meansand to gain map data according to a photographed image in said cameraand 



a means to display gained map data with a photographed image in said camera. 



[Claim 5] A system which presents an image photoed with a camera including a 
subject which movescomprising: 

A position identification means for it to be attached to said subject and to identify 
a three-dimensional position of the subject. 

A dispatching means which disseminates acquired position identification 
information. 

A detection means to detect a three-dimensional positiona three-dimensional 
postureand a field angle of a camera which takes a photograph. 
A storing means which stores three-dimensional map data. 

A means to carry out transparent transformation of the three-dimensional map 
data stored in this storing means based on a detection result of said detection 
meansand to gain map data according to a photographed image. 
A means to display gained map data with a photographed image including said 
subject in said camera. 

[Claim 6]Claim 1an image presenting system given in any of 4 or 5 they arewherein 
said camera is a camera which performs aerial photography. 

[Claim 7]Claim 1an image presenting system given in any of 4 or 5 they arewherein 
said camera is a camera which takes a photograph underwater. 
[Claim 8]The image presenting system according to any one of claims 1 to 7 
having further a means to record data of a detection result of said detection 
meansand data of a photographed image in said camera. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the image presenting system which 
is made to superimpose the map information of the photographing location on a 
photographed imageand is shown especially about the image presenting system 
which presents the photoed image. 
[0002] 

[Description of the Prior Art]In the Television Sub-Division (TV) report in the 
Great Hanshin-Awaji Earthquake which took out serious damage. Since the 
overland route was cut offthe relay program which the journalist of a broadcasting 
station who incorporated the aerial photography image and got into [ the pilot of 
an aircraft or an aircraft ] with the camera carried in aircraftsuch as a light 
airplane and a helicopterand announcer report that the situation of the spot is was 
in use. And many televiewers have held big concern and were watching such a 
relay program broadcast in real time. 

[0003]Howeverin these the aerial-photographing reports of a series ofshortage of 



the information for pinpointing photographing iocationssuch as the name of a place 
and a directionand a mistake were often able to see. For examplethe scene where 
the journalist who acts as intermediaryand announcer corrected confusedly the 
information on the name of a place reported previouslya directionetc. in response 
to advice of the pilot of a helicopter or the staff of the office side was notably 
seen especially by the aerial-photographing report at night. 
[0004] 

[Problem to be solved by the invention]Various unfavorable conditions — that the 
building which serves as a mark as a cause which such a phenomenon generated 
for the interruption to service in the smoke by that the city scene has changed 
according to the earthquake disaster and a fire or the night was not able to be 
checkedand the on~the~spot person in charge was unfamiliar to aerial 
photographing — can be considered. Howeverin the initial stage after the 
earthquake which the urgent situationssuch as escape and rescueconcentrate 
even if it is what kind of causeEven if it thinks from the influence of media called 
TVit of it being a serious problem which may influence many human lives is certain 
for such a false report to compare and to flow through a public electric wave for a 
moment also. 

[0005]As long as such a problem has one of the important missions of TV report in 
the instancy nature of information and judgment of the position information on the 
relay in the spot is left to the interpretation of a small number of relay staffit can 
be said that it exists universallywithout restricting to an unprecedented disaster 
like the Great Hanshin-Awaji Earthquake. 

[0006]By the wayGPS (Global Positioning System)Although it is possible for it to 
be used for the navigation of the route of a subjectthe overland routeand the sea 
routeetc.to be able to acquire easily the exact position information on the real 
time of a subjectand to carry out tracking of the locus of a subject on a map 
screenThe information for recognizing the situation around real time visually is not 
providedbut there is a problem of being dearth of information in an emergency. 
[0007]This invention is made in view of this situationand is a thing. 
The purpose is to provide the image presenting system which can prevent the 
false report at the time of the relay using a photographed image by displaying map 
information on an image in piles in real time. 

[0008]Other purposes of this invention are to provide the image presenting system 
which superimposes the exact position information over a photographed image on 
a photographed imageand can display it. 

[0009]The purpose of further others of this invention is to provide the image 
presenting system which can perform the localization in search of a subject and 
tailing easily by superimposing map information on the photographed image of the 
subject which moves. 
[0010] 

[Means for solving problem]The video display system which this invention requires 
for Claim 1 is characterized by that the system which presents the image photoed 



with the camera comprises: 

A detection means to detect the three-dimensional positionthree-dimensional 

postureand field angle of the camera which takes a photograph. 

The storing means which stores three-dimensional map data. 

A means to carry out transparent transformation of the three-dimensional map 

data stored in this storing means based on the detection result of said detection 

meansand to gain the map data according to a photographed image. 

A means to display the gained map data with the photographed image in said 

camera. 

[001 1]The camera with which an aircraft is equipped with the video display system 
concerning Claim 2A detection means to prepare for this aircraft and to detect the 
three-dimensional positionthree-dimensional postureand field angle of this 
cameraThe storing means with which said aircraft is equipped and which stores 
three-dimensional map dataPrepare for said aircraft and transparent 
transformation of the three-dimensional map data stored in this storing means is 
carried out based on the detection result of said detection meanslt has a means 
to gain the map data according to the photographed image in said cameraand a 
means for said aircraft to be equipped with and to display the gained map data 
with the photographed image in said camera. 

[0012]The camera with which an aircraft is equipped with the video display system 
concerning Claim 3A detection means to prepare for this aircraft and to detect the 
three-dimensional positionthree-dimensional postureand field angle of this 
cameraThe transmitting means with which said aircraft is equipped and which 
transmits the data of the photographed image in said cameraand the data of the 
detection result of said detection means on the groundThe reception means which 
rece j ves the send data from this transmitting meansand the storing means which 
stores three-dimensional map dataBased on the detection result of said detection 
meanstransparent transformation of the three-dimensional map data stored in this 
storing means is carried outand it has a means to gain the map data according to 
the photographed image in said cameraand a means to display the gained map data 
with the photographed image in said camera. 

[0013]The camera with which the video display system concerning Claim 4 is 
formed in the regular position enabling free rotationA detection means to detect 
the three-dimensional posture and field angle of this cameraand the storing means 
which stores three-dimensional map dataBased on the detection result of said 
detection meanstransparent transformation of the three-dimensional map data 
stored in this storing means is carried outand it has a means to gain the map data 
according to the photographed image in said cameraand a means to display the 
gained map data with the photographed image in said camera. 
[0014]The video display system which this invention requires for Claim 5 is 
characterized by that the system which presents the image photoed with the 
camera including the subject which moves comprises: 

A position identification means for it to be attached to said subject and to identify 



the three-dimensional position of the subject. 

The dispatching means which disseminates the acquired position identification 
information. 

A detection means to detect the three-dimensional positionthree-dimensional 
postureand field angle of the camera which takes a photograph. 
Based on the detection result of said detection meanstransparent transformation 
of the three-dimensional map data stored in the storing means which stores 
three-dimensional map dataand this storing means is carried outA means to gain 
the map data according to a photographed imageand a means to display the gained 
map data with a photographed image including said subject in said camera. 

[0015]The video display system concerning Claim 6 is set they to be [ Claim 1 and 
any of 4 or 5 ]and said camera is characterized by being a camera which performs 
aerial photography. 

[0016]The video display system concerning Claim 7 is set they to be [ Claim 1 and 
any of 4 or 5 ]and said camera is characterized by being a camera which takes a 
photograph underwater. 

[0017]The video display system concerning Claim 8 is set they to be [ any of 
Claims 1-7 ]and it has further a means to record the data of the detection result 
of said detection meansand the data of the photographed image in said camera. 
[0018]In the image presenting system of this inventiona detection means detects 
the photographing condition of a cameratransparent transformation of the three- 
dimensional map data applicable based on the detection result is carried outthe 
map data for a display is obtainedand when displaying a photographed image with a 
camerathe map data is superimposed and displayed. Thereforethe information 
value of a camera image can be raised. In this inventionthe false report of the 
name of a place and the direction which could display simultaneously the map 
information of the building used as the name of a place of citiestowns and villages 
and a boundaryand a marketc. on the photographed image in real timefor 
examplehad been generated in the former in the relay report at the time of the 
disaster outbreak using an aircraft is prevented. It is quite obvious and a 
televiewer can be provided with the position information on the photography spot. 
[0019]The camera to photo may be a camera for aerial photographing which is 
carried in aircraftsuch as a light airplanea helicopterand a balloonand moves with 
an aircraftand may be a fixed point camera fixed to buildingssuch as a skyscraper 
and a towerand may be a camera which was formed underwater and which carried 
out waterproof specification further. 

[0020]In addition to a camerasaid detection means is carried in an aircraftsaid 
storing meansthe acquisition meansand the displaying means are installed on the 
groundthe detection result in imaging data and a detection means with a camera is 
transmitted on the ground from an aircraftand it may be made to see the 
photographed image of the on-the-spot photo superimposed on map information 
on the ground. Or if all said detection meansstoring meansthe acquisition 
meansand displaying means are carried in the aircraft in addition to the camerait is 



possible to see inside an aircraft the photographed image of the on~the~spot 
photo superimposed on map information. Of courseit cannot be overemphasized 
that the inside of an aircraft and terrestrial all may be equipped with said storing 
meansan acquisition meansand a displaying means. 

[0021]Other image presenting systems of this invention are further provided with a 
position identification means for it to be attached to a subject which moves in 
addition to a detection meansa storing meansan acquisition meansand a displaying 
means which were mentioned aboveand to identify a three-dimensional position of 
the subjectand a dispatching means which sends acquired position information. If 
position information on a subject which moves is sentan aircraft will be searched 
for a subject according to the position informationthe existence will be 
checkedand a subject will be followed after that. If it does in this wayvisual search 
of a subjecttailingand localization are possible on a photographed image of an on- 
the-spot photo superimposed on map information. 

[0022]In these image presenting systemsit has further a means to record 
photographed image data in a cameraand a detection result of said detection 
means. If it records and leaves information on a detection resultand data of a 
photographed imageit is available as sources of information of next various 
analysis processings, 
[0023] 

[Mode for carrying out the invention]Hereafterthis invention is concretely 
explained with reference to Drawings in which the embodiment is shown. 
[0024] Drawing 1 is a mimetic diagram showing an outline of an image presenting 
system of this inventionand the sensor 32 containing GPS for detecting a three- 
dimensional positiona three-dimensional postureand a field angle of the camera 31 
for photography and the camera 31 which presents an unstable actiona 
gyroscopea field angle encoderetc. is carried in the helicopter 30, While the 
helicopter 30 fliesvarious kinds of camera information detected by picture image 
data obtained with the camera 31 and the sensor 32 is transmitted towards the 
ground one by oneand it is received by the receiver 34 of the ground station 33. 
Camera information is sent to the CG (Computer Graphics) unit 35. In the CG unit 
35transparent transformation of the three-dimensional map data memorized 
beforehand is carried out based on the camera informationand CG map image data 
is obtained. Obtained map image data is sent to the display 36. Received 
photographed image data is also sent to the display 36. And the display 36 
performs image display in a mode which made map information superimpose on a 
photographed image of an on-the-spot photo obtained with the camera 31, 
[0025]GPS which incorporated the communication functionfor example is attached 
to the car 37 which runs the groundand the data of the three-dimensional position 
information on the car 37 is sent to the ground station 33. If the camera 31 carried 
in the helicopter 30 is rotated and the car 37 exists in the photographing areathe 
position of the car 37 will be displayed on the display 36. Thenif RIKOPUTA 30 is 
flown in the direction of the car 37it is possible to be able to recognize the car 37 
visually in the on~the~~spot photo image superimposed on map informationand to 



carry out tracking of the car 37. 

[0026]Nextthe example of the image presenting system of this invention is 
explained in full detail. 

(A 1st embodiment) Drawing 2 is a mimetic diagram showing the example of 
composition of the image presenting system (a 1st embodiment) of this invention. 
The component member in the image presenting system of this invention is divided 
into the component carried in aircraftsuch as a light airplanea helicopterand a 
balloonand the component provided in a terrestrial broadcasting station. 
[0027]In an aircraftthe camera 1 and the gyro sensor 2 for three-dimensional 
posture detection of the camera 1The encoder 3 for field angle detection of the 
camera land the signal converter 4 which changes the detecting signal of the gyro 
sensor 2 into posture dataThe signal converter 5 which changes the detecting 
signal of the encoder 3 for field angle detection into field angle value dataGPS 
device 6 for the detecting positions of the camera 1the antenna 7 for GPS 
receptionthe GPS error correction equipment 8the computer 9 for controlmonitor 
TVIOthe data encoder 1 land the sending set 12 are carried. On the other handthe 
receiving set 21 the data decoder 22the host computer 23the storage 24 of an 
inside or the exteriorthe recording equipment 25and the video editing device 26 
are formed in the terrestrial broadcasting station. 

[0028]The camera 1 is a common TV camera or a video camera for broadcastand 
sends out the video signal produced by carrying out aerial photography to monitor 
TV10 and the data encoder 11. 

[0029]The gyro sensor 2 for detecting the three-dimensional posture is attached 
to the camera land the gyro sensor 2The head angle (the direction of the 
northsoutheast and west of the camera 1) of the camera 1a helix angle (inclination 
to the sliding direction of the camera 1)and a bank angle (inclination to the 
longitudinal direction of the camera 1) are detectedand the detecting signal is 
outputted to the signal converter 4. The signal converter 4 changes a detecting 
signal into HPB data (H(Head) : the direction data of the camera 1 inclination data 
of the sliding direction of the P(Pitch):camera 1 inclination data of the longitudinal 
direction of the B(Bank):camera 1)and outputs it to the computer 9 for control. As 
a gyro device which detects the posture of such a camera 1TANS Vector™ of the 
gyroscope mini dt by DATA TEC Co. Ltd. and Trimble Japanetc. can be usedfor 
example. 

[0030]The encoder 3 for field angle detection is attached to the zoom ring 
attached to the lens of the camera 1the angle of rotation of the zoom lens of the 
camera 1 is detectedand a detecting signal is outputted to the signal converter 5. 
The signal converter 5 changes a detecting signal into FV data (FV (Focusing 
Value): focal distance of the lens of the camera Dand outputs it to the computer 9 
for control. As an encoder device which detects the field angle value of the zoom 
lens of such a camera 1the lens position detector by KOSUMEITO can be usedfor 
example. 

[0031]Near the camera 1GPS device 6 which is a position sensing device using an 
artificial satellite currently used widely by car-navigation system is formedand 



GPS device 6It asks for a three-dimensional absolute position of the camera 1 
according to an input signal (GPS data) from the antenna 7and XYZ-DR-DT data 
(XYZ: position data of the camera Imove direction data of the DRcamera 1data of 
DT:date and time) is outputted to the computer 9 for control. In this casethe GPS 
error correction equipment 8 has received correction data of GPS dataand GPS 
device 6 amends an input signal from the antenna 7 based on this correction data. 
As a GPS device which detects a three-dimensional position of such a camera 1a 
Sierra™GPS chip set of IPS-3000 by Sony Corp. and Trimbleetc. can be usedfor 
example. The GPS error correction equipment 8 can suppress an error of GPS 
data which are ability ready for receiving in broadcast service or walkie™ 
talkiessuch as a cellular phone and FMand tend to become large according to a 
weather condition etc. about periodical transmission service to 1 m or less. 
[0032]A camera position and a posture metering device consist of the above gyro 
sensors 2the encoder 3 for field angle detectionthe signal converter 4the signal 
converter 5GPS device 6the antenna 7and the GPS error correction equipment 8. 
And each data from a camera position and a posture metering device is 
transmitted to the computer 9 for controland an integrating process is carried out 
there. 

[0033]The computer 9 for control computes the parameter of the viewpoint in the 
three-dimensional map space of imagination in a computer from the parameter 
(XYZ dataHPB dataFV data) obtained with the camera position and the posture 
metering device mentioned above. The parameter of the view position to compute 
is defined by seven dimensions of viewpoint coordinates (XYZ)a visual line 
direction (the head angle Hhelix-angle Pbank angle B)and a field angle (FV) like the 
location parameter of the camera 1. The computer 9 for control carries out 
transparent transformation of the three-dimensional map information memorized 
inside according to the parameter result of the computed view positionand sends 
out the changed map information (a directiona citiestowns and villages nameand a 
boundary) to monitor TV10. As software for carrying out transparent 
transformation of this three-dimensional map datathe software by a plus one 
company "VPX" can be usedfor example. 

[0034] Monitor TV10 builds in the video editing deviceit carries out image display in 
the mode which made the map information from the computer 9 for control 
superimpose on the image acquired with the camera land is shown to the crew 
member of an aircraft. The time which presenting of map information takes 
| ncreases j n proportion to the data volume of the map information processed. 
Thereforethe map information sent out to monitor TV10 is selected if needed so 
that real time nature may not be spoiled. For this reasonthe edit function which 
was excellent in dialogism using GUI (Graphical User Interface: graphical user 
interface) etc. is required. 

[0035]The computer 9 for control sends out the parameter data (XYZHPBFV) of a 
view positionand above-mentioned DR-DT data to the data encoder 11 with the 
form of serial data. The data encoder 1 1 codes the serial data from the computer 
9 for controland the picture image data from the camera land outputs the coded 



data to the sending set 12. The sending set 12 changes coded data into a radio 
wave signaland transmits it to a terrestrial broadcasting station. 
[0036]The computer 9 for control has the calibration software and sector-of™ 
calibration software for each components which carried in the aircraftand is 
performing initial setting of the gyro sensor 2 and the encoder 3 grade for field 
angle detectionand a sector of calibration. 

[0037]The receiving set 21 receives the radio wave signal transmitted from the 
sending set 12and outputs it to the data decoder 22 and the recording equipment 
25. The data decoder 22 decodes coded data to picture image datathe parameter 
data (XYZHPBFV) of a view positionand DR-DT dataPicture image data is sent out 
to the video editing device 26and the parameter data (XYZHPBFV) and DR~DT 
data are sent out to the host computer 23. As this data encoder 1 1 / data 
decoder 22product VXmade from pro Spa electron-50 T/R can be usedfor 
example. 

[0038]The host computer 23 computes the parameter of the viewpoint in the 
three-dimensional map space of imagination in a computer from the inputted 
parameter data (XYZHPBFV). The parameter of the view position to compute is 
the same as the case of the computer 9 for control mentioned above. The host 
computer 23 the three-dimensional map information memorized to the external 
storage 24 which consists of CD-ROMHDan MO disketc.lt sends out to the video 
editing device 26 by using map information (a directiona citiestowns and villages 
name and a boundarya buildinga railroada roada rivera lakethe summit of the 
mountain) acquired by carrying out transparent transformation according to the 
parameter result of the computed view position as CG picture image data. As 
software for carrying out transparent transformation of this three-dimensional map 
datathe software by a plus one company "VPX" can be used like the computer 9 
for control. 

[0039]The video editing device 26 compounds the picture image data a photograph 
of was taken on the spot from the data decoder 22and CG picture image data in 
which the map information from the host computer 23 is shownand outputs the 
synthetic video data to an image sending device or VTR. The recording equipment 
25 records the signal received with the receiving set 21 on videotape etc. 
[0040]Nextoperation is explained. Firstinitial setting of the gyro sensor 2 and the 
encoder 3 grade for field angle detection is performed by the calibration software 
of the computer 9 for control. The ground is photoed from the air with the camera 
land the video signal is sent out to monitor TV10 and the data encoder 11. During 
the photographythe posture of the camera 1 is detected with the gyro sensor 
2and HPB data is outputted to the computer 9 for control from the signal 
converter 4The field angle of the zoom lens of the camera 1 is detected with the 
encoder 3 for field angle detectionFV data is outputted to the computer 9 for 
control from the signal converter 5and DT data in which the time at the time of 
DR data in which the move direction of XYZ data and the camera 1 which shows 
the three-dimensional position of the camera 1 is shown furtherand photography is 
shown is outputted to the computer 9 for control from GPS device 6. 



[0041 ]In the computer 9 for controlthe parameter of the viewpoint defined by 
seven dimensions in the three-dimensional map space of imagination in a 
computer is computed based on the parameter data (XYZ dataHPB dataFV data) 
obtained from these camera positions and posture metering devices. According to 
the parameter of the computed viewpointthe map information (a directions 
citiestowns and villages nameand a boundary) from which transparent 
transformation of the three-dimensional map information memorized inside was 
carried outand it was acquired from the viewpoint is sent out monitor TV10. And 
the picture which superimposes this map information on the image photoed with 
the camera 1 is displayed on monitor TV10. 

[0042] Drawing 3 is a mimetic diagram showing an example of the picture displayed 
on monitor TV10. To the photographed imagethe sign of the direction which shows 
northsoutheast and westthe name (A city and B town) of citiestowns and 
villagesand the boundary (boundary of A city and B town which shows with an 
alternate long and short dash line) of citiestowns and villages are overlappedand 
are displayed. And the journalist of a pilot or a broadcasting station relays the 
situation of the spotlooking at the picture superimposing of this monitor TV 10. 
Since monitor TV10 shows a pilot or a journalist a picture which put the 
transparent map on actual sceneryeven if not detailed to the geography of the 
spotthe geographic information which was wrong in them at the time of relay is not 
reported. 

[0043]Parameter data (XYZHPBFV) and DR-DT data of a view position are sent 
out from the computer 9 for control to the data encoder 1 land are changed into 
coded data. The coded data is outputted to the sending set 12and is changed into 
a radio wave signal which can communicateand the radio wave signal is 
transmitted to a terrestrial broadcasting station. 

[0044]At a terrestrial broadcasting stationit is received by the receiving set 21 
and a radio wave signal from an aircraft is changed into the original coded data. 
The coded data is changed into the original picture image dataparameter data 
(XYZHPBFV) of a view positionand DR-DT data which decoding processing was 
performed and were photoed by the data decoder 22. 

[0045]In the host computer 23parameter data (XYZHPBFV) is inputted and a 
parameter of a viewpoint in three-dimensional map space of imagination in a 
computer is computed based on this. According to a computed parametermap 
information from which transparent transformation of the three-dimensional map 
information memorized to the external storage 24 was carried outand it was 
acquired from the viewpoint is sent out to the video editing device 26. 
[0046]The original picture image data is sent out from the data decoder 22 to the 
video editing device 26the picture image data and map information from the host 
computer 23 are compoundedand the synthetic video data is outputted to an 
image sending device or VTR. This synthetic video data is recorded with VTR. 
[0047]The video signal equivalent to this synthetic video data is outputted as a 
televising signal of TV-for-home JON from an image sending device. And the 
picture according to this synthetic video data is displayed on Television Sub- 



Division of each home. Drawing 4 is a mimetic diagram showing an example of the 
picture displayed on domestic Television Sub-Division. In addition to the sign of a 
directionthe name of citiestowns and villagesand the boundary of citiestowns and 
villagesthe information on the name (D mountainE mountainF riverG bay) of a 
building (C pons) and geographical feature is also superimposed and displayed to 
the photographed image. 

[0048]The signal containing the picture image datathe parameter data of a 
viewpointand DR-DT data which were received with the receiving set 21 is sent 
and recorded also on the recording equipment 25. The recorded data is used when 
performing behind many-sided analysis processing. For examplethe processing etc. 
which create the map showing the situation at the time of photography from the 
image a photograph of was taken on the spot can be considered. Since data 
required for analysis processing is once recorded on videotape etc. of courseit is 
not necessary to perform such analysis processing in real time at the time of 
photography. 

[0049]In the above image presenting systemsa three-dimensional positiona 
postureand a field angle of the camera 1 of an unstable action which perform aerial 
photography are measured in real timeSince a picture of three-dimensional map 
data by which transparent transformation was carried out based on the result is 
doubled with a photographed image and displayed in real timea situation currently 
photoed can be transmitted without an error. Since coded data of a three- 
dimensional position and a posture of the camera 1 at the time of photographyand 
a field angle is recorded in real time with a photographed imagethese information 
can be effectively used at the time of analysis of a next photographed image. 
[0050]This image presenting system as an object for the report of various 
disasterssuch as an earthquakea forest fireflood damagea typhoonetc. with a local 
spreadto a televiewer. It not only can provide more exact information in real 
timebut it can contribute it to uses other than for reports dramatically as an early 
exact information gathering means for a synthetic disaster information system with 
which establishment will be desired from now onor an information gathering means 
for grasp of a damage situationand a measure against relief. 

[0051]Hereafterin this inventionprocessing operation of software in the computer 
9 for control and the host computer 23 with high importance is explained. Drawing 
ELand drawing 6 are flow charts which show a flow of the processing 
operation drawing 5 shows an initialization action and drawing 6 shows actual 
processing operation (operation from data input of the camera 1 to drawing 
processing of map data). 

[0052]Firstinitial motion is explained with reference to drawing 5 . If a program 
startsafter initializing various variables (S1)the map data (city databuilding 
dataetc.) of the area which is the photography spot will be read (S2). 
Subsequentlyafter initializing the data of the camera 1 (S3)the coordinates of a 
re f erence point are inputted (S4). Hereas a reference pointpoints in which the 
coordinates are knownsuch as a departure point or a specific point in the spotare 
adopted. Nextthe timer for specifying the sampling period of the data input from 



the camera 1 is set up (S5). In this stateit will be in the state waiting for an event 
of the data input of the camera land initial motion will be ended. 
[0053]Nextactual processing operation is explained with reference to drawing 6 . It 
is detected as the event of the data input of the camera 1 occurring (S11)and 
anticipation coordinates are calculated from input data 1 second before saving 
(S12). Anticipation coordinates are position coordinates of the camera 1 expected 
when only the time which drawing processing of next map data takes passes. The 
anticipation coordinates searched for are compared with the last anticipation 
coordinatesand it is judged whether the difference is in an error span (S13). In 
being outside an error spanit judges that there were abnormalities in Measurement 
Division of a camera position and a posture metering device to the camera land 
the anticipation coordinates are not used for drawing processing of next map 
databut it carries out a return to it as it is. 

[0054]On the other handin being in an error spanit changes the anticipation 
coordinates into CG coordinates (S14). And the input data of the present camera 
1 is saved as input data 1 second before next time (S15). Subsequentlythe event 
of CG picture-image-data drawing is generated (S16)the map data nearest to a 
viewpoint is searchedand a sort list is created (S17). After clearing the contents of 
a screen display and the Z-uffer which stored the information on a far and near 
relation (S18)drawing processing is performed based on map data (S19). a position 
with the drawing processing near a viewpoint to a wire frame — geographical 
feature — in written formit carries out to drawing by drawing and a wire frameand 
a building is performed for drawing and an area name in order of drawing and 
drawing of geographical feature by a polygon of a building by a polygon. Howeverif 
the event of the data input of the camera 1 occurseven if it is on the wayit will 
end compulsorily (S20) and the return of the drawing processing will be carried out. 
[0055]In the processing operation mentioned abovesince he is trying to draw map 
data based on the anticipation coordinates of the camera 1the time lag 
accompanying drawing processing can be compensated and CG map data which 
agreed correctly in the photographed image of the camera 1 can always be 
displayed. Since drawing processing will be promptly terminated compulsorily if the 
event of the data input of the camera 1 occurspresenting of map information is 
possible at the rate of the animation average. 

[0056](A 2nd embodiment) it is possible to see an on-the-spot photo image 
superimposed on map information in a 1st above-mentioned embodiment also 
monitor TV10 in an aircraft — having made — although — in such an 
examplethere is a problem that equipment carried in an aircraft increases in 
number. Since a situation of the spot can be visually checked when a person 
detailed to geography of the spot has got into [ an aircraft jit is common for a 
display of an on-the-spot photo image superimposed on map information to be 
unnecessary. Thereforean example to which the ground carried out base office 

of the display of an on-the-spot photo image superimposed on map 
information is an image presenting system of a 2nd embodiment of this invention. 
Drawing 7 is a mimetic diagram showing an example of composition of this image 



presenting system (a 2nd embodiment). 

[0057]In drawing 7 the same number is given to drawing 2 and identical partsand 
those explanation is omitted. In composition of a 2nd embodimentmonitor TV is 
not formed in an aircraft but an on~the-spot photo video signal from the camera 1 
is sent only to the data encoder 1 1. The computer 9 for control only unifies each 
data from a camera position and a posture metering device which comprises the 
gyro sensor 2the encoder 3 for field angle detectionthe signal converter 4the 
signal converter 5GPS device 6the antenna 7and the GPS error correction 
equipment 80peration which acquires map information like a 1st embodiment is 
not performed. 

[0058]In drawing 7 the host computer 23CG picture image data in which the map 
information acquired based on the parameter data of the photographing condition 
which built in the image compositing device and was decoded by the data decoder 
22 is shownThe on-the-spot photo picture image data decoded by the data 
decoder 22 is compounded with the host computer 23the synthetic video data is 
outputted to monitor TV27and monitor TV27 performs image display in the mode 
which made map information superimpose on the image of an on-the-spot photo. 
[0059]Although a 1st and 2nd embodiment mentioned above explained the case 
where the camera 1 was carried in an aircraftit may be made to use a fixed point 
camera which is attached to the predetermined place of buildingssuch as a 
skyscraper and a towerpivotable. In such a casesince the three-dimensional 
position of a camera is immobilizationGPS is unnecessary and the encoder for 
picture detection same as a detection means to detect the three-dimensional 
posture and field angle of a camera as the case of the rotary encoder and aircraft 
which are replaced with a gyro sensor is required for it. 

[0060](A 3rd embodiment) Drawing 8 is a mimetic diagram showing an example of 
composition of an image presenting system (a 3rd embodiment) of this invention. 
The 3rd embodiment is an example which uses objects which movesuch as human 
beingan animalvehiclesa marine vesseland an airplaneas a subjectand was made to 
perform tracking of the subject. 

[0061 ]In drawing 8 18 and 19 are a GPS device and a transmitterand the locator 20 
which unified these GPS devices 18 and transmitters 19 is attached to a subject 
which tracking objectssuch as human beingan animalvehiclesa marine vesseland an 
airplanemoveGPS device 18 asks for an absolute position of a subjectand sends 
the position data (XYZ data) to a base station with the transmitter 19. The same 
number is given to the same component member as a 1st and 2nd embodiment 
mentioned aboveand those explanation is omitted. 

[0062]Nextoperation is explained. As a 1st and 2nd embodiment also explainedthe 
image photoed with the camera 1 carried in the aircraft is displayed on monitor 
TV27 in the state where it was superimposed on map information. Firstin searching 
such a state for a subjectas much as possiblewith a wide-angle lensit is made to 
rotate 360 degrees and photos the camera 1 in it in the air. Since the position 
data is sent to the base station from the locator 20 attached to the subjectif a 
subject exists in the photographing area of the camera 1the position and 



attribution information of the locator 20 which were attached to the subject will be 
displayed on monitor TV27 of a base station. Nextif an aircraft is moved in the 
direction of the locator 20 displayed on monitor TV27 and it zooms ina target 
subject can be discovered in the on-the-spot photo image superimposed on the 
map information displayed on monitor TV27. 

[0063]Thusit is possible to follow the subject which movesrecognizing a subject 
visually in the on-the-spot photo image superimposed on map informationif 
movement of an aircraft and operation of a camera are performedso that the 
position of the locator 20 may not be missedafter looking for and discovering a 
subject. 

[0064]The on-the-spot photo picture image datathe parameter data (XYZHPBFV) 
of a photographing conditionand DR-DT data in the search and tailing processing 
to such a subject are recorded on videotape etc. with the recording equipment 25. 
Thusif it setsit can use as sources of information of the analysis processing of 
movement of a subject. 

[0065] Drawing 9 is a mimetic diagram showing the example of composition of 
modification of the image presenting system of a 3rd embodiment. Monitor TV10 is 
carried also in the aircraft and monitor TV10 in an aircraft enables it to perform 
the display with the position of the locator 20and attribution informationand the 
display of a subject like terrestrial monitor TV27 like a 1st embodiment in this 
modification. 

[0066]Although we decided to carry out tracking of the one subject which moves 
in the explanation mentioned aboveit is also possible to carry out tracking of two 
or more subjects simultaneously by a 3rd embodiment. 

[0067]Although map information is superimposed on the photographed image in the 
air and it was made to display on it in each embodiment mentioned abovelf above- 
mentioned camera 1gyro sensor 2encoder 3 for field angle detectionand GPS 
device 6 are made into waterproof specificationmap information is superimposed 
on it andof coursethe photographed image acquired underwater as well as each 
embodiment mentioned above can be displayed on it. 

[0068]Although map information is superimposed on a photographed image and it 
was made to display on it in each embodiment mentioned abovea display screen is 
dividedand a photographed image may be displayed on one screen areaand it may 
be made to display map information on the screen area of another side. 
[0069]By the wayin the information presenting system of this inventionas map 
information superimposed on a photographed imagelt adds to basic datasuch as a 
citiestowns and villages boundarya transportation networka public facilitya main 
buildinga landmarkthe summit of the mountaina river namelt doubles with a users 
purpose and the option data of urban equipmentsuch as gasa waterworkselectrical 
and electric equipmentand a highwaya tourist routea marathon coursea 
construction planan underground installationhistorical ruinsetc. is prepared 
according to a user's usage. Thusby preparing the map information of various 
sortsthe use area of the information presenting system of this invention is 
substantially expandable. 



[0070]Hereafterthe field of the invention of this invention is explained. As a user of 
the system of this inventionan individual place of businessand a public institution 
are objects. The 1st field of the invention of the system of this invention is use 
which raises the information value of the air or an underwater photographed image 
using the composition of the 1st or 2nd above-mentioned embodiment. Since 
various map information can be superimposed on the photographed image of an 
on-the-spot photo in real timethe information value of the present airan 
underwater photographed imageor a fixed point camera image can be raised by 
leaps and bounds. That isthe screen where it was superimposed on the map 
information about the image with the image can give a user worthy informationand 
can change TV as a mere video service means to TV (Augmented TV) as an 
information service means including an image. 
[0071]As an example of such 1st field of the inventioninformation 
gatheringsurveillanceand analysis of a disaster (disasters with especially local 
breadthsuch as an earthquakeflood damagea mudflowand a forest fire) — in 
additionUse by TV broadcast (hobby programssuch as a sport relay broadcast 
programtourist resort inquiryetc. of a relay report program and marathon at the 
time of a disasteretc.a yacht raceetc.)There are use (real-time presentation of a 
plan institutionetc.) in a display and control of maintenance of equipment of gasa 
waterworksetc. which were laid undergroundand a large-scale construction 
planetc.and broad use is possible. 

[0072]The 2nd field of the invention of a system of this invention is use which 
performs visual tracking using composition of the 3rd above-mentioned 
embodiment. If a mobile for tracking has a locatorsearch of the mobiletailingand 
localization can be visually performed on TV image superimposed on map 
information. Broad use is possibleeven if scientific research of sporting event 
management of accident rescue of operation management of vehicles and a marine 
vesselmountainsand marine human beinga yacht racean automobile rallyetc. and 
action of wildlifeetc. occur as an example of such 2nd field of the invention and it 
is in this 2nd field of the invention. 
[0073] 

[Effect of the InventionjAs mentioned abovewith the video display system of this 
inventtonsince the image on which the image of the on-the-spot photo with a 
camera was made to superimpose map information can be shownthe information 
value of the photographed image in a camera can be raised. Exact tracking of the 
object which moves can be performed. Since the photographing condition of 
photographed image data and a camera is recordedthese information can be 
effectively used at the time of the analysis of a next photographed image. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is a mimetic diagram showing the outline of the image presenting 



system of this invention. 

[Drawing 2] It is a mimetic diagram showing the composition of an example of the 
image presenting system of this invention. 

[Drawing 3] It is a mimetic diagram showing the display example of the picture in 
monitor TV carried in the aircraft. 

[Drawing 4] It is a mimetic diagram showing the display example of the picture in 
Television Sub-Division for home use. 

[Drawing 5] It is a flow chart which shows the processing operation (initialization 
action) of the software in the image presenting system of this invention. 
[Drawing 6] lt is a flow chart which shows the processing operation (drawing 
processing operation of map data) of the software in the image presenting system 
of this invention. 

[Drawing 7] lt is a mimetic diagram showing the composition of other examples of 
the image presenting system of this invention. 

[Drawing 8] It is a mimetic diagram showing the composition of the example of 
further others of the image presenting system of this invention. 
[Drawing 9] It is a mimetic diagram showing the composition of the example of 
further others of the image presenting system of this invention. 
[Explanations of letters or numerals] 

1 Camera 

2 Gyro sensor 

3 The encoder for field angle detection 
6 GPS device 

9 The computer for control 

10 Monitor TV 
12 Sending set 

18 GPS device 

1 9 Transmitter 

20 Locator 

21 Receiving set 

23 Host computer 

24 Storage 

25 Recording equipment 

26 Video editing device 

27 Monitor TV 



